Sorry if I am misunderstanding. But was the intent of the original issue to be able to validate maps before persisting? If so will this not ignore the validation

Good point, we will need to carry those accordingly and also document the feature, since this is specific to apply changes, we don’t handle changesets in maps otherwise.

@azizk Do you want to take that on? If not we can probably merge this (but Jose please keep me honest if this alone is not ok to go out) and I could tackle the other stuff.

Yeah, I suspected something else was missing to make this a complete feature. I think I'd rather hand it over to you. Too many functions to look at. 😅

Thanks!

No worries. Thank you for this change. @josevalim are you ok to merge just this one for now?

Or maybe I am getting ahead of myself. I have doubts now which one of these 2 we are going with

1. we only ever make the change for apply_changes and document it
2. We also try to do it for persistence. I.e resolve the value or report an error during repo operations

the second point sounds like a good feature to me but I haven’t evaluated whether it’s possible

The best would be likely to support this everywhere, because this solves the problem of us supporting nested schemaless changesets, but I am unsure how practical it is, especially if it will require us special casing map types everywhere.

special casing map types everywhere.

Sounds like introducing a schema 😉

I haven't come across the need to do nested schemaless changesets. I would think if you reach for nested schemaless, you should probably define a schema because it's about to get complex.

One thing I can see is if you have one or a few functions that do some basic validations on maps that can be reused. Like a function that takes in a list of keys and checks if they are present. As opposed to creating separate schemas for every type of map you might want to do this on.

sorry @azizk. so to close this out would you be able to add into the docs for apply_changes/1 that we allow this kind of schemaless changes? and then we'll take it away to see if it makes sense to extend it further

@greg-rychlewski

One thing I can see is if you have one or a few functions that do some basic validations on maps that can be reused. Like a function that takes in a list of keys and checks if they are present. As opposed to creating separate schemas for every type of map you might want to do this on.

That's a good example. I'd like to mention another one that came to me yesterday. It's about saving storage space.

I have been working on a work project dealing with OpenRTB. I have an embedded schema for the Bid object which has a lot of fields. When I store a Creative schema I only want to store a limited number of fields from the Bid embedded schema. So I define field :bid, :map instead of embeds_one :bid, Bid, because otherwise Ecto stores all properties from Bid even if the values are nil. I still want to validate the bid map field in the Creative changeset function to be sure and to avoid future surprises.

What I do atm is to use validate_change/3 and add an error if the Bid changeset is invalid.

Initially we wanted to allow schemaless changesets, but maybe it's worth thinking about allowing any kind of changeset. Otherwise it becomes necessary – in this case at least – to convert a schema changeset to a schemaless changeset (I guess by doing Map.update!(changeset, :data, &Map.from_struct/1?).

PS: I didn't want to open a new issue, so commented here, if that's okay.

@azizk Jose had a good idea where custom types can be utilized. I'm not sure if you saw this before but you could basically make your own type and control these things through the cast callback. And you could even make the type parameterized to make it generalizable.

Let us know what you think:

https://hexdocs.pm/ecto/Ecto.Type.html
https://hexdocs.pm/ecto/Ecto.ParameterizedType.html

As one example, you could get very close to mimicking cast_embed by making a parameterized type like MyValidatedMap that takes a :with parameter. That parameter is a function that does all the changeset functions and then as an extra step at the end checks if the changeset is valid and then applies the changes or returns an error.